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Abstract: The influence of neutrosophy in the previous period is constantly growing in many 
areas of science and technology. Moreover, various applications of the neutrosophic approach have 
become more common in recent years. Our goal in this research is to utilize the neutrosophy to 
improve the performance of the Dai-Liao conjugate gradient (CG) method. Specifically, in this 
research, we propose and investigate a new neutrosophic logic system to calculate the key parameter 
t involved in the Dai-Liao CG iterations. Theoretical analysis and numerical experience indicate that 
the efficiency and robustness of the new rule for determining t. Combining the neutrosophy and 
the Dai-Liao conjugate gradient method, we propose and explore a new Dai-Liao CG iterations for 
solving large-scale unconstrained optimization models. The global convergence is established under 
common assumptions and the backtracking line search. Finally, by conducting numerical experiments, 
computational evidence demonstrates that the new fuzzy neutrosophic Dai-Liao conjugate gradient 
method is computationally effective and robust. 
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1. Introduction and background results 


Numerous iterative methods have been developed to solve the large-scale uncon- 
strained optimization problem 


minimize f(x), x € R", (1) 


in which f : R” — Risa continuously differentiable function and bounded below. Contin- 
uing well established notation, g, = g(x.) = Vf (x;) stands for the gradient vector of f at 
the actual iterative point x;, and further yz_1 = gx — ge_1 and Sp_1 = xx — Xz_1. Utilizing 
the extended conjugacy condition 


di yr-1 = —tgp8k_1, t>0, (2) 


Dai and Liao in [1] suggested the conjugate gradient (CG) iterations 


Xk1 = X_ + apd, (3) 
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where the x; is the last calculated iteration, x, is a new iterative point, a; is a positive 
step size parameter defined as the output of an inexact line search, and d, is a descent 
direction. The search directions {d,, k > 0} are created by the recurrent regulation 


_ _ 
d,={ 8% , . 
‘ { —g.+prrdy1, k>1, (4) 


where BP" is the CG coefficient which describes the type of CG method by the general rule 


T. 
DL 8Yk-1 
k 


SA Sk—1 
T t c ij 
dy Yk-1 di Yk-1 


(6) 


wherein ¢ > 0 is an appropriate scalar. The Dai-Liao (DL) method guarantees global 
convergence for uniformly convex objective functions. These results have attracted a lot of 
attention, leading to the creation of several methods based on various patterns for defining 
fx. Most of these methods were developed by modifying the conjugate gradient parameter 
BP" [2-9]. For more details, see the survey on the DL family of nonlinear CG methods in 
[10]. One of rules for defining 6; is denoted as pynseh and defined in [7] by 


pMHSDL = Si Yk1 8, Sk-1 (6) 
qt sYe-1 We Ve-1 
lgell 


such that ¥;_1 = gr — Ter 11 8k-1 and t > 0 is as in (5). 


Due to the large influence of the size tf on numerical results generated by the DL class 
of CG methods [11], one of the most common issues is the determination of an appropriate 
value t. We can distinguish two research directions based on the previous results in 
determining proper values t in the DL established CG iterations. The first direction of 
research consists of a group of DL methods that aim to find a suitable constant value for t 
[1,2,6-8] during iterations, while the second direction consists of a group of DL methods 
that propose a suitable control in recalculating ft in each iteration. In this research, we 
prioritize the second research stream: find values t that changes appropriately accros 


iterations. The quantity t determined in kth iterative step will be denoted by t,’’, where i is 
a variant of the algorithm for defining ft. 

Some of the most important adaptive choices for the DL parameter t; will be presented 
in the rest of this section. Hager and Zhang in [12,13] suggested the CG-DESCENT method, 


which is classified into the group of the DL CG methods (5) defined by t = 1) and 


2 
1) — 2 Wyenall (7) 
Yx_18k-1 


Dai and Kou suggested DK method in [14] where the CG coefficient BPX is of the form 


T 2 T T 
k— = V1 8k-1 Sk 
pbk = 8. Yk-1 (: llyx—11l k-1 \# 1 (8) 


T . 
Yp_-1 4-1 Yp_18k-1—*(USx—1ll?_ / dE yea 


In the equality (8), the parameter %; is defined utilizing the self-scaling memoryless BFGS 
method. It is also obvious from (8) that the DK method is involved into the DL CG class of 


methods where t = 4) is defined by 


2 Tg 
42) og Mye—all® _ Ya-18e-1 i) 
k T 2 
Yx—18k-1 I|$x—1| 
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Babaie-Kafaki and Ghanbari in [15] proposed the subsequent two rules for calculating ¢ in 
(5): 


T 
(3) — Sx—1¥k-1 | llye—all 
t (10) 
lls<—all?_——|ISx—al 
= yell 
(4) Yk-1 
+4) _ : (11) 
Ilsx—1l 


Andrei in [16] originated a new DL class, denoted DLE, where t = 1{5) is defined by 


(5) Si_1Yk-1 


: (12) 
[sea ?? 
A special place in the DL iterations is occupied by the DL method with 
2 
6 


vst Yet 1¥k-— 1 
where v > i is a constant, has been defined according to the sufficient descent condition 
1 ay < — 2 Wk>0 14 
8 Ak S—C |lgxl|", Vk 2 0, (14) 


such that c > 0 is a constant independent of the cost function convexity and the line search 
rules (for more details see [17]). 
Lotfi and Hosseini in [18] suggested the subsequent rule 


Pamala}, (15) 
where 
ms (1 —Mallgx ir) SE 18k + 5t ae Hie Bx—11I" Il Sx— ill? 
te — , (16) 
gi Sk—1+ oe * ‘lee ill" lsx—1|I? 

Ss ee 
iy =C+maxd EVA gigs yr (17) 

Ilsx—1| 


and C,r are positive constants. 
Ivanov et al. in [19] proposed a variant of the Dai-Liao CG method (6), known as the 


Effective Dai-Liao (EDL) CG method, where t = 48) is determined as 


2 
max{1.df ge} + (maxfo, 48} 41) gel? 


The experiments performed in [19] verify that the EDL iterations outperform many existing 
CG variants. 


The basics of neutrosophy. Neutrosophic logic was applied in [20] in regulating 
proper step sizes for a class of accelerated gradient-descent optimization methods. The 
approach in [20] assumes an additional fuzzy parameter which stabilizes the behavior of 
an important class of gradient-descent family. Motivated by that approach, in this research 
we apply neutrosophy in order to enhance performances of DL methods. Based on the 
review and analysis of the class of DL methods, we propose a new method for determining 
tz. The proposed method defines t, as the output produced by an appropriate neutrosophic 
logic controller (NLC). Our idea is to replace the classical parameter t, by an adaptive 
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neutrosophic logic parameter 1, determined as the output of the NLC. Since 0 < th <1 
our decision is to define t; as the value of 1%, without additional parameters. 

A fuzzy set theory utilizes a membership function (MF) Ty(A) € [0,1],A € A in the 
universe A that defines the degree of membership of A in ¥ C A [21]. The intuitionistic 
fuzzy set (IFS) ¥ is established using both degrees of membership and non-membership 
function Ty (A), Fy (A) € [0,1],A € A [22], which are mutually corelated by Ty(A),Fy(A) : 
A — [0,1] and 0 < Ty(A) +Fy(A) < 1. The IFS theory was originally generalized 
in [23,24] by the the neutrosophic theory. The background of the neutrosophic logic is the 
utilization of the indeterminacy I(A). In that direction, entries of a neutrosophic set are 
determined by three independent MFs [23,24]: the truth-MF T(A), the indeterminacy-MF 
I(A), and the falsity-MF F(A). Due to the indeterminacy-MF, the neutrosophic logic is 
based on the symmetry involved in the ordered triple of MFs (T, 1, F) and the inequality 
0 <T+I1+F < 3 if all three MFs are independent. Clearly, T is the symmetric pole to 
its opposite pole F with respect to I, which represents an axis of the symmetry between 
T and F [25]. The same observation is valid for refined neutrosophic set that assumes 
the refined indeterminacies I, and Iz between T and F [26]. The MFs of a neutrosophic 
set ¥ satisfy Ty (A), ly (A), Fy (A) : A > [0,1], which which based on their independence 
implies 0 < Ty(A) + Iy(A) + Fy(A) < 3, and enables a symmetry between them. In [27], 
the authors originated a neutrosophic-based multiple criteria decision-making procedure 
based on previously introduced symmetry measure. 

The benefits of the NL approach over the FL and IFL are discussed in [20]. 

Motivation and highlights of main results. Our task in this paper is to improve the 
behavior of DL class for solving unconstrained nonlinear optimization problems with the 
support of an appropriate neutrosophic logic system. The principal results obtained in this 
paper are presented as follows. 


(1) We examine the application of NL in determining the parameter t in the Dai-Liao 


CG method (5). 

(2) A theoretical analysis is accomplished to confirm the global convergence of the 
proposed method. 

(3) |Anumerical comparison is given between the proposed FDL algorithm and other 
known DL algorithms. 


The sections of the paper are arranged as follows. Introduction, motivation and a 
brief review of obtained results are presented in Section 1. A neutrosophic-based control 
for defining appropriate changeable values tf, is proposed in Section 2. Moreover, we 
present details of the FDL method. The global convergence behavior of the FDL method 
is examined in Section 3. Numerical comparison of the FDL method with main standard 
DL methods is presented in Section 4, and a comparison with some known variations of 
the DL class of methods, is also given. Final conclusions are presented in the concluding 
section. 


2. Fuzzy neutrosophic Dai-Liao conjugate gradient method 


The fuzzy neutrosophic Dai-Liao CG method is defined as a modification of the Dai- 
Liao CG method (3), where the search directions {d;} are calculated by the recurrence 
rule 


- k=0 
d,-{ ~8” 19 
: { go. + Pd, KEI, 9) 
where the CG coefficient BFP" is defined by 
a T 
k-1 Sk-1 
FDL _ 8x. 8k (20) 


T kaT , 
dy 1 ¥k-1 qe Yk-1 
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such that 1% := 1% is a proper fuzzy neutrosophic parameter. Our intention is to define 1; 
as a function of A; := f(x) — f (Xk41) ie., VE = V¢(Ax). More precisely, v;,(A;) is defined 
subject to the following constraint 


0<v4(Ak) <1. (21) 
It is know that v%.(A,) = 0 reduces (2) into 
dj yr-1 = 0. (22) 


Hence, the equation (22) can be considered as a reflection of the conjugacy condition, which 
in conjuction with (4) determines the HS parameter [28] 


SLY k-1 


prs — 
‘ di 1Yk-1 


(23) 


Alternatively, for v;(A;) = 1, the equation (2) is considered as a conjugacy condition that 
implicitly satisfies the quasi-Newton characteristics. For more details on these cases, see 
[1,10]. 

The idea for defining a new parameter tk in the Dai-Liao CG method (5) comes from 
the neutrosophic logic. According to this decision, our intention is to define t, = vy(Ax) 
inside the interval {0, 1] according to neutrosophic principles. 

The generic layout of the fuzzy neutrosophic Dai-Liao CG method is given in the 
diagram in Figure 1. 


fe Gir Uk, Xz 
Fuzzy neutrosophic Dai-Liao (FDL) eas 
conjugate pradientinetied Backtracking line search 


Output: f, Xp44 


Figure 1. The global structure of the fuzzy neutrosophic Dai-Liao CG method. 


The input of the NLC presented in Figure 1 is Ay := f (xx) — f (x,41) and the output 
is the desired step size vy. This means that our basic idea is to define 1, based on two 
consecutive values of the objective function f. On the other hand, the backtracking line 
search is reponsible for appropriate step lengths a, in (3) and then the descent direction dx 
by (19). Using 1% it is possible to compute pee in (20). Finally, (3) generates new iterative 
point x41. 

To develop the FDL method, it is necessary to plan three global steps: neutrosophisti- 
cation, neutrosophic inference engine, and de-neutrosophistication (score function). 


(1) Neutrosophication maps the input A, := f(x) — f(x,11) into neutrosophic ordered 
triplets (T(A;),1(Ax), F(A;)). The MFs are defined with the aim to improve the CG 
iterative rule exploiting numerical experience. The sigmoid function with the slope 
defined by ¢, at the crossover point A = ¢» is a proper choice for T: 


T(A) = 1/(1+e78(4-82)), (24) 
A proper choice for F is the following sigmoid function: 


F(A) = 1/(1 + e81(4-82)), (25) 
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The Gaussian function with the standard deviation ¢, and the mean c» defines the 
indeterminacy I: 
_ (A=69)? 
I(A)=e 1. (26) 
Then the neutrosophication of A € R is defined as the transition A + (T(A),I(A), F(A)), 
where the MFs are determined in (24)-(26). 


(2) Neutrosophic inference between an input fuzzy set J and an output fuzzy set is based 
on the subsequent “IF-THEN” regulations: 


R, fT =Sp—> O = {TF} 
Ry EI =Sy — O = {T LF}. 


Fuzzy sets Sp and Sy point, respectively, to positive or negative errors. Applying 
the unification # = It, U ko, we define DO; = Jo R;, i = 1,2, where o denotes the 
fuzzy transformation. In addition, for a fuzzy vector € = {T(A,),I(Ax), F(Ag)}, it 
follows KzoR(C) = Ksor, V K3oR, = Sup(K3 A Kp,), i = 1,2, where / and \ denote 
the (min, max, max) and (max, min, min) operator, respectively. In this research, the 
centroid defuzzification method is utilized to generate a vector of crisp outputs (* € 


3, 
= px — So Skson()de 
Ji 9 KIoR (C)dg 
(3) De-neutrosophication is based on the transformation (T(A,),1(A,),F(Ax)) + vy, € R 
resulting in a crisp value 1% and suggested as: 


Ug = 2 — (T(Ag) + 1(Ag) + F(Ax))- (27) 


The diagram of Figure 2 presents the NLC based on the neutrosophic rules. 


IF-THEN 
Rules 


i . 
—Eean i 


Neutrosophic 
Inference 


Engine 


Actions 


Neutrosophication 
De-Neutrosophication 


Mi 


Figure 2. The NLC design based on the neutrosophy. 


The settings in the NLC employed in numerical testing are arranged in Table 1. 


Table 1: Recommended parameters in NLC. 


Set Membership Function C1 C2 Weight 
Sigmoid function (24) 1 3 1 

Input Sigmoid function (25) 1 3 1 
Gaussian function (26) 120 0 1 

Output Score function (27) - - 1 


Our imperative requirement is 0 < v;,(A,) < 1, requested in (21). This statement is 
verified in Lemma 1. 


Lemma 1. The inequality (21) holds for the given choice of the score function (27) and the parame- 
ters given in the Table 1. 
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Proof. In order to prove (21), we need to replace the MFs (24), (25), and (26) in (27). After 
applying the parameters from Table 1, we get 


2 
Ak 


T(Ag) = 1/(1+e°@-9)), F(A) = 1/(1 +4083), 1A) =e BF 


Elementary calculation gives 


A2 


vp(Ag) = 2 — (T(Ag) + 1(Ax) + F(Ag)) = 1 — e7 280. (28) 


A careful analysis of the function (28) inside the interval A, € (—0co,+00) discovers 
min v;,(A;,) = 0 and max %(A,) = 1, which proves the inequality (21). 


Graphs of T(A;),1(A,), F(A,) are displayed in Figure 3(a). The fulfilment of the re- 


quirements (21) in the NLC output 1; generated throughout the described de-neutrosophication 


is illustrated in Figure 3(b). 


|| —2-(T(A,)+(A,)+F(A,) 


-400 -300 -200 -100 0 100 200 300 400 


(a) Neutrosophication. (b) De-neutrosophication. 


Figure 3. Neutrosophication (24)-(26) and de-neutrosophication (27) guided by the parameters in 
Table 1. 


Remark 1. The objective function decreases with the flow of iterations and tends to the minimal 
value, which means limk+o0 Ag = 0, i-€., limp sco VE (Ag) = 0. Such behavior leads to vy, — 0 as 
the minimum of f approaches, so the impact of the proposed neutrosophic strategy decreases and 
disappears, which agrees with our goal. 


Remark 2. Obviously, larger values of A, lead to increasing values v;,.(A,) approaching to 1, which 
will be denoted as v¢(Ax) 7 1. In addition, based on the limit A, — 0, we anticipate smaller 
values vz (Ax) approaching 0, i.e., vy(Ag) \, 0 in final iterations. As a result, v,(Ax) is suitable as 
an adjustable regulator for the quantity t in the Dai-Liao CG method. 


The backtracking line search from [29] begins from a =1 and generates further step 
sizes which ensure decrease of the goal function in each iteration. Algorithm 1, restated 


from [30], is used to define the primary step size a. 


Algorithm 1 The backtracking line search. 


Input: Objective function f(x), foregoing point x;, the search direction d;, a real positive 
constants 0 < » < 1,and0 <w < 0.5. 
1€=1. 
2: While f(x; + €dx) > f (xp) + wlgt dy, do ¢ := Lg. 
3: Output: a, = &. 
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Algorithm 2 of the FDL method is described as follows: 


Algorithm 2 Fuzzy neutrosophic Dai-Liao (FDL) conjugate gradient method. 


Input: A starting point x9 and 0 < €,6 <1. 
1: Assign k = Oand dp = —Zo- 
2: If 


Lf (x41) — F (xx) 
i+lfeor =" 


IIgxl| Se and 


STOP; 
else go to Step 3. 


: (Backtracking line search) Regulate a; € (0,1] utilizing Algorithm 1. 
: Calculate xp41 =x, + apd. 

: Calculate 9441, Yk =Sk41 — Sk Sk =Xk41 — Xk- 

: Calculate Ay := fk — fest: 

: Calculate T(A;),1(A,), and F(A;) as in (24)-(26). 

: Calculate v% := vj, (Ax) using (27). 


: Calculate Se by (20). 


10: Generate dy41=—gk41 + ey dy. 
11: Setk := k +1, and go to Step 2. 


wo AN D oO FF WwW 


3. Convergence Examination 


The subsequent assumptions are necessary during the theoretical examination of the 
FDL algorithm. 


Assumption 1. (1) The level set 0 = {x € R"| f(x) < f(xo)} of the iterative process (3) is 
bounded. 

(2) The objective f is continuously differentiable in a neighborhood P of U with the Lipschitz 
continuous gradient g. Such assumption initiates the existence of a constant L > 0 such that 


Is(u) —g(v)|| < Llu—v|, Vuve P. (29) 
The Assumption 1 provides the existence of quantities Y and y such that 
ju-—v|| <Y, VuwEeP (30) 


and 
Ig(u)|| <7, Vue P. (31) 


If Assumption 1 holds, in view of the uniform convexity of f, there exists 0 > 0 
satisfying 
(g(u) — g(v))"(u—v) > Allu—v|?, Vuv €U, (32) 


or equivalently, 
flu) > fv) +9(w)"(u—v) + $llu—vI?, uv ed. (33) 
It follows from (32) and (33) that 
St 1Yk-1 > Olisxall? (34) 


and F 
f (x1) — f 0%) > -B (xe) "Sea + 5lisx—1Il. (35) 
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By (29) and (34), one concludes 
Allsx—all? < sf_a¥K—1 S L|lsx-1l|?, (36) 
and further @ < L. 
The inequality (36) implies 
T _ Ad 
S,_1Vk—1 = Xe—-14_1VK-1 > 0. (37) 
Taking into account #,_; > 0 and the last inequality, we conclude 
di_iyr—1 > 0. (38) 


Lemma 2. [31,32] Let the constraints in Assumption 1 hold and the points {x;} be produced by 
the iterations (3)-(4). Then the following inequality is satisfied: 


©. Iigillt 
< +oo, 39 
de Tae? o 


Lemma 3. Observe the suggested Fuzzy neutrosophic Dai-Liao CG method defined by (3), (19), 
(20). If the search procedure guarantees (38), for allk > 0, then 


gid < —cllgx | (40) 
for some c > 0. 


Proof. In the initial stage it follows g}do =— ||go||*. Since c = 1, obviously (40) is satisfied 
in the initial stage k = 0. Assume (40) for some k > 1. Applying the inner product of both 
the left and right hand side in (4) with Si, it is concluded 


84k =—|gKll? + Bee’ ge dea 
=~ lige? (et " fo gla. 
=—li? + ar Bede sla (41) 
mote Bg a 
221g, \2 SLY k-1 1g, pes Beda)” 


di iVe-1 dt yK-1 


Using (21) in conjunction with (38) and a,_; > 0, we conclude 


; a1 (gp dk—1)* 


> 0. (42) 
di-1Yk-1 
Now from (41) and (42) it follows 
T 2 SEY k—1 T 
8 A < —|I[Bxll° 4 a ay BE okt 
k-17K— (43) 


— —heell? (4g aye—1)* + (geyn—1) (Bp de—1) (Ae 1¥K—1) 
(dt_y¥«—1)? 
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Applying the inequality P™Q < }((||P\|? + |Q||?) to the equality (43) with P = 
5 (Ak_1Vk—-1) 8k and Q = V2(gid;_1)yx_1, it is obtained 


Sk 


(dT sye—1)? + 3(IF5(4E yea) Bel? + I V2(8f4x1)¥e-1IP) 


T 
BAK S 
‘ (dt iye—1)? 
— —ligell2(4t_ aye)? + 3 (3 GE yea llgell? + 20821 lly aI?) 
(dt y¥e—1)? 
= [Igell? (4g ¥K—1)? + (Ay yK—1)* Iigell? + (ge de—1)" Ilya II? 
(dt yyn1)? 
= Hep? + Lee? (g¢ 4-1) Ilya? 
4 (df syk—-1)? 
J Nige 1 ae ae (gf des) llyx—a |? 
i 4 (a, 14f_1¥K-1)? 
1 og Neill? lde—all? Ilye—1 I? 
<—lgxl? + zllgell? + > 3 (44) 
($,_1¥K-1) 
1, 9, Wgell7ex—14e—a117 Ilye—a ll? 
< 
See gan 2 IIs—14 
1 2, llgell7ilse—all7Ilye—a ll? 
< 
en “sae ls. 
1, 2, Wgell7ilye—all? 
ra 
Se De Pls. al 
1) vo, Lilgell*iise—all? 
2 
a vee ae 5.11 
1 ge 5 
=— [gel + 7 lge 72 IlSxll 


=-(1- 


jar) lel? =- (3-5 
q ge NBN a 


The requirement (40) is satisfied for c = (3 — ir) 


lIgxl?. 


in (44) and an arbitrary k > 0. 


Theorem 1 confirms the global convergence of the FDL flow. 


Theorem 1. Let the constraints in Assumption 1 be valid and f be uniformly convex. Then the 
series {x;} generated on the basis of (3), (19) and (20) fulfils the limit relation 


lim inf ||g|| =0. (45) 
k-0o 


Proof. Suppose the opposite. Since (45) is not valid, we conclude the existence of c, > 0 


satisfying 


IIgxll| = cr, for 


Squaring both sides of (19) one derives 


all k. (46) 


lldill? = ligell? — 2Bp* sede + (Bow)? IIde—all?- (47) 
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Taking into account (20), we obtain 
T T 
SK Yk-1 8x Sk-1 T 
— 2B, gp dx_1 =—2 VK 8 Ak-1 
di 1Yk-1 dt 1Yk-1 
T T 
SYk-1 7 Xr -18,Ak-1 7 
=—2 8 Ak—1 — Vk =e 1 (48) 
( di Yet dt iYk-1 
ae ca 2 
Si Yk-1 7 a. —1(8_ A_1) 
=—2 >. nt 77 dp_1 — Ve : 
( dt 1Yk-1 
Now from (42), it follows 
T 
8 Yk-1 = 
~2p gfe 1 <2] BP \igtay | < 2tellye al gia, 
k-1Yk-1 lay ye 
7 5 kill Bell? lhye—a| Il dix—1| 
O10 py Vk—-1 
= I sicll7l¥%—a11Mse—1l (49) 
St_1Yk-1 
Z 9 liseli? Elsi—1l| Ilsx—1l 
~ 6||sx_1 ||? 
Oi ais 
a Gy lise . 
Now, an application of (20) initiates 
FDL _ SEY k-1 8, Sk-1 a SLY k-1 am UpQpSk—1 
By dz har di 
k—-1¥k-1 k-1Yk-1 k-1¥k-1 
< BiYk-1 — VASE Sk—1 ae \SiYk—1 — V4.8; Sk—1| 
> di 1YK-1 e140 1Yk-1 
_ \SiYk—1 = VES Sk—1| 
Xk-1 T 
S,_1Yk-1 
253 |SaYe—1 — VeBgSe—1| lSi (Ye—-1 — VeSk—1)| 
oe 8||sx—1 | Ook ||dx—a|]? (50) 
ec sell yx—all + vellse—aI) 
~ Bax _4||dx—1I? 
ec Wexll(Lise—all + ve) lIsx—all) _ geil (2 + ve) Ilse—al 
= Box 4 ||Ax—1||? Boex—1||dx—1 ||? 
— Ilgell(L + v1 Ilde_sl 
Boe. 1 ||dx_1||? 
— (b+ ve) leet 
8\|dx_1| 
Further (21) and (50) lead to 


~  Olldx—al| 
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Using (49) and (51) in (47), we get 


QL) a, (b+ 1? lee? 
d Qe 2 2 d,_ 2 
a < lee? + liga? + arp a 
2L L+1)*||gxl|? 
= |lgel? + Sigel? + E+ Heel 
_2L . (L+1)? 5 
= (14 SE) sul co) 


@+2L (L+1)? 4 
=(— az — ) lies 
(0+2L)6+(L+1)? 
= 62 IISx 


|”. 


Dividing both sides of (52) by ||gx||* and utilizing (46), it can be decided 


dill? (@+2L)0+(L+1)* | 1 
Igxll*~ 6° cy 
4 2a (53) 
Iskll” Onc 
ld, l2 = (6+ 2L)0+(L+1)?° 
The inequalities in (53) imply 
= ligll* S 6? ct 
| 
do ial? = Le @+2L)0+ (E41)? ~ ee 


Therefore, ||g;|| > cy; causes a contradiction with Lemma 2. 


4. Numerical Results 


In this section, numerical results obtained by the FDL method are analyzed and 
compared with the numerical results generated by the EDL [19] and DL [1] methods. 


All the algorithms were written in Matlab R2017a and ran on a 64-bit Lenovo laptop 
(Intel Core i3 2.0 GHz, RAM 8 GB) with the Windows 10 operating system. The implemen- 
tation of the FDL method is based on Algorithm 2, while EDL and DL implementation are 
based on algorithms given in [19] and [1], respectively. 


The numerical testing is performed on 50 test functions collected in [33,34], with 
dimensions from the range [100,20000]. All three tested methods used start from the 
same initial point xo for each test function. Each case in testing is evaluated 10 times with 
gradually increased dimensions n = 10,5 x 107, 10°,3 x 103, x10%,7 x 10°,8 x 10%, 104, 
1.5 x 104 and 2 x 104. 


The uniform terminating criteria for observed DL, EDL, and FDL algorithms are 


lf (xka1) — f Ox) | ee Se 
1+ [fea <6, e€=10°,6=10 ”. 


We are going to evaluate the efficiency of the FDL method and compare it with the 
EDL and DL methods under the backtracking search based on the parameters w =0.0001 
and g=0.8. 


Summary numerical results for DL, FDL, and EDL methods, performed on 50 test 
functions, are shown in Table 2, where “Test function’, ‘Nitr’, ‘Nfe’, and “Icpu’ represent 
the name of the tested function, the total number of iterations, the total number of function 
evaluations, and the running time, respectively. 


IIgkll <€ and 


To visually compare the performance of tested methods, we used the performance 
profiles technique [35] on numerical results corresponding to Nitr, Nfe, and Tcpu criteria 
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Table 2: Summary numerical results on unconstrained problems of DL, FDL, and EDL methods for the Nitr, Nfe, and Tcpu. 


Test function > Le : EDE : EDE 
Nitr/Nfe/Tcpu Nitr/Nfe/Tcpu Nitr/Nfe/Tcpu 
Extended Penalty 1905 /77578/32.438 1610/62534/24.266 2304/82602/39.344 
Perturbed Quadratic 14555 /606750/379.359 10800/440213/206.5 10012/408474/248.5 
Raydan 1 4337 /114595/98.984 5497 /122843/76.813 4194/109164/96.938 
Raydan 2 1427 /2864/3.188 67/144/0.281 2572540 /5145090/894.453 
Diagonal 1 5809 /223750/245.578 5488 /212491 /227.781 4673 /178295 /219.109 
Diagonal 3 5247 /196745 /423.766 4531/168162/307.594 4596 /171636/366.203 
Hager 1742/31516/103.672 1242 /22799 /47.063 1940/33206/98.766 
Generalized Tridiagonal 1 2058 /32313/49.5 2160/32033/27.5 2161 /33285/44.703 
Extended Tridiagonal 1 310/2932/8.391 182/2501/6.297 308 /4129/12.766 
Extended TET 1140/9840/11.031 619/5808/5.484 749 /6362/5.969 
Diagonal 5 1394/2798 /6.938 60/130/0.609 3053907 /6107824/3124.875 
Extended Himmelblau 50/2431/1.016 51/2602/0.813 50/2413/0.938 
Perturbed quadratic diagonal 1837/69156/18.453 1261/36785/13.875 2157 /86977 /34.797 
Quadratic QF1 13895 /526995 /187.313 21989 /846402/376.156 10199 /379554/122.844 
Extended quadratic penalty OP1 1080/17440/9.922 1524/23840/8.25 1157/18043/9.109 
Extended quadratic penalty QP2 218/9479/11.047 112/5513/4.953 218/9194/8.906 
Quadratic QF2 19211/847031/348.781 18861 /816310/225.891 15555 /689736/250.891 
Extended quadratic exponential EP1 1254/3443 /3.172 56/404/0.516 21431 /43829/7.531 
Extended Tridiagonal 2 22468 /998473 /549.484 3668 /114169 /87.438 10989 /510713/93.609 
TRIDIA (CUTE) 33278 /1647913/967.234 40156/1977068/950.547 29133 /1428866/675.422 
ARWHEAD (CUTE) 1624/81625/44.875 1529 /72379 /31.594 1219/57140/28.672 
Almost Perturbed Quadratic 14904./621925/259.797 19675 /829784/357.359 13201 /543372/188.047 
LIARWHD (CUTE) 30/2705/1.281 30/2732/1.25 30/2739/1.438 
POWER (CUTE) 532442 /44419504/16742.672 580790/48609979/17435.609  629342/52431424/23630.781 
ENGVAL1 (CUTE) 2489 /33103/13.781 2400/32299/10.719 1975 /27260/12.922 
INDEF (CUTE) 21/1924/2.125 26/2238/2.5 30/2610/4.266 
Diagonal 6 1583/3197 /4.531 74/185 /0.359 7052401 /14105032/5037.219 
DIXON3DQ (CUTE) 320921 /1775846/1083.281 229757 /1368033/727.172 257451 /1517252/1045.328 
COSINE (CUTE) 20/1600/1.891 20/1697/1.891 20/1700/2 
BIGGSB1 (CUTE) 249919 / 1400798 /832.375 259475 /1549293 /810.766 236612 /1389720/945.672 
Generalized Quartic 866/11273/3.984 1099/8951 /4.063 959 /10662/3.125 
Diagonal 7 1453 /4564/6.875 68/162/0.469 469477 /940686/140.172 
Diagonal 8 1371/3962/5.359 67/199/0.422 594522 /1193760/195.094 
Full Hessian FH3 2237 /6202/7.125 52/513/0.688 767988 / 1537759 /188.469 
Diagonal 9 3312/138545 /225.719 5344/217150/224.906 4520/189307 /260.453 
HIMMELH (CUTE) 20/1690/4.797 20/1758/4.531 20/1760/4.891 
FLETCHCR (CUTE) 303212/10189775/5073.688 300227/10011849/4704.125 289670/9702961/4411.453 
Extended BD1 (Block Diagonal) 1597 /16783/7.625 1227/15639/5.875 1200/12605/6.625 
Extended Maratos 72/3366/1.188 50/2069/0.719 40/1975/0.75 
Extended Cliff 234/2992/2.078 217/6000/4.891 950/13187/6.188 
Extended Hiebert 70/7215/1.938 70/7220/1.828 70/7228 /1.859 
NONDIA (CUTE) 33/3066/1.375 30/2829/1.266 32/3031/1.625 
NONDQUAR (CUTE) 58/4652/18.047 45 /3666/17.219 86/4989/19.016 
DQDRTIC (CUTE) 3456 /87105/26.453 2327 /59047/16.406 3637 /92315/34.953 
Extended Freudenstein and Roth 1376/46597 /10.734 3390/111830/28.516 2018 /66654/16.172 
Generalized Rosenbrock 282948 /8410218/4125.516 280440 /8335396 /4088.547 281792 /8373946/4055.172 
Extended White and Holst 76/5794/9.219 50/3171/7.281 59 /4022/11.563 
Extended Beale 118/6791/14.047 72/3118/5.906 181/4748/6.75 
EG2 (CUTE) 507 /29388 / 47.547 697 /48512/119.875 811/39769 /122.469 
EDENSCH (CUTE) 1694/23160/89.453 2089 /27821/83.266 1684/22731/116.844 


generated by DL, FDL, and EDL methods. An upper graph in a performance profile 
corresponds to the method that shows better performance. The vertical axis of each 
performance profile in figures undicates the percentage of test functions for which the 
considered method is the winner between compared methods, whereby the right-hand 
side corresponds to the percentage of successfully solved test functions. 
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Figures 4 and 5 plot the performance profiles for the data in Table 2. Graphs in Figure 
4 illustrate the performance profiles Nitr and Nfe for DL, FDL, and EDL iterations based 
on the data from Table 2. In Figure 4(a), it is noticeable that DL, FDL, and EDL methods 
are able to solve all tested functions, wherein the FDL method produces the best results 
in 54.0% (27 out of 50) of test functions compared with DL (26.0% (13 out of 50)) and EDL 
(38.0% (19 out of 50)). From Figure 4(a), it is observed that the FDL graph reaches the 
top first, so FDL is the best relative to other examined methods with respect to the Nitr 
criterion. 


Figure 4(b) indicates that the FDL graph is the most efficient and successfully solves 
all test cases. In addition, the obtained numerical results confirm that FDL performs well in 
most cases. Most specifically, FDL is the fastest because it solves about 48.0% (24 out of 50) 
of tested functions with the least Nfe compared to the DL and EDL methods. Meanwhile, 
the DL and EDL are superior for solving 22.0% (11 out of 50) and 30.0% (15 out of 50) test 
functions, respectively. Hence, the numerical behavior of FDL is superior compared to the 
DL and EDL methods for the given test functions. 


(a) Nitr performance profile. (b) Nfe performance profile. 


Figure 4. Performance profile for DL, FDL, and EDL methods. 


Figure 5 shows Tcpu performance profiles graphs of DL, FDL, and EDL methods. 
It is observable that DL, FDL, and EDL are able to solve all tested functions. Further 
examination leads to the conclusion that the FDL method is the best in 74.0% (37 out of 
50) of the test cases compared with DL (12.0% (6 out of 50)) and EDL (16.0% (8 out of 50)). 
Analysing the graphs in Figure 5, a clear conclusion is that the FDL graph comes to the top 
first, which confirms its dominance in terms of Tcpu. 


on 


Figure 5. Tcpu performance profile for DL, FDL, and EDL methods. 
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By analyzing the results shown in Figures 4,5 and in Table 2, we conclude that the 
FDL method achieved better results. This observation leads us to the final conclusion that 
the proposed FDL method is effective for solving unconstrained optimization problems in 
terms of all three criteria (iterations, function evaluations, and processor time). 


5. Conclusion 


In current research, we propose a novel approach to determining the parameter t 
in the Dai-Liao CG iterations. New approach is based on finding suitable values for a 
non-negative parameter f in the DL method using neutrosophic logic. Utilizing t = v;(A,) 
in (5), an original strategy in defining the Dai-Liao CG parameter Bf" is proposed and a 
novel fuzzy neutrosophic Dai-Liao (FDL) CG method is presented. 

Numerical experiments and comparisons with some well-known CG methods and the 
theoretical convergence analysis show effectiveness of the method. The numerical testing 
and initiated comparison are based on standard performance profiles, such as the total 
number of iterations (Nitr), the total number of function evaluations (Nfe), and the running 
time (Tcpu) performances for each function and each method. Analysis of the obtained 
numerical results revealed that the FDL method is the most efficient. 

We are convinced that the obtained results will be a motivation for further research in 
defining improved DL methods strengthened by the neutrosophic logic. 

Future scientific research in this direction can be continued in in several directions. 
Previous research has shown the effectiveness of the neutrosophic principle in scaled 
gradient descent methods and DL class of CG methods. The challenge is to apply sucha 
principle to other non-linear optimization methods. On the other hand, there is a wide 
variety of different possibilities for defining the principles of neutrosophication and de- 
neutrosophication, which can be considered in future research. Finally, there is a great 
opportunity in improving the Neutrosophic inference engine used in this research. 
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